This is the supplementary analytic output for the paper Social Thermoregulation: A Meta-analysis by IJzerman et al. 

It reports detailed results for all models reported in the paper. The analytic R script by which this html report was generated can be found on the project’s OSF page at: [LINK].


Brief information about the methods used in the analysis:

RMA results with model-based SEs k = number of studies; sqrt in “Variance components” = tau, the standard deviation of true effects; estimate in “Model results” = naive MA estimate

RVE SEs with Satterthwaite small-sample correction Estimate based on a multilevel RE model with constant sampling correlation model (CHE - correlated hierarchical effects - working model) (Pustejovsky & Tipton, 2020; https://osf.io/preprints/metaarxiv/vyfcj/). Interpretation of naive-meta-analysis should be based on these estimates.

Prediction interval Shows the expected range of true effects in similar studies. As an approximation, in 95% of cases the true effect in a new published study can be expected to fall between PI LB and PI UB. Note that these are non-adjusted estimates. An unbiased newly conducted study will more likely fall in an interval centered around bias-adjusted estimate with a wider CI width.

Heterogeneity Tau can be interpreted as the total amount of heterogeneity in the true effects. I^2$ represents the ratio of true heterogeneity to total variance across the observed effect estimates. Estimates calculated by two approaches are reported. This is followed by separate estimates of between- and within-cluster heterogeneity and estimated intra-class correlation of underlying true effects.

Proportion of significant results What proportion of effects were statistically at the alpha level of .05.

ES-precision correlation Kendalls’s correlation between the ES and precision.

4/3PSM Applies a permutation-based, step-function 4-parameter selection model (one-tailed p-value steps = c(.025, .5, 1)). Falls back to 3-parameter selection model if at least one of the three p-value intervals contains less than 5 p-values. For this meta-analysis, we applied 3-parameter selection model by default as there were only 11 independent effects in the opposite direction overall (6%), causing the estimates to be unstable across iterations. pvalue = p-value testing H0 that the effect is zero. ciLB and ciUB are lower and upper bound of the CI. k = number of studies. steps = 3 means that the 4PSM was applied, 2 means that the 3PSM was applied.

PET-PEESE Estimated effect size of an infinitely precise study. Using 4/3PSM as the conditional estimator instead of PET (can be changed to PET). If the PET-PEESE estimate is in the opposite direction, the effect can be regarded nil. By default (can be changed to PET), the function employs a modified sample-size based estimator (see https://www.jepusto.com/pet-peese-performance/). It also uses the same RVE sandwich-type based estimator in a CHE (correlated hierarchical effects) working model with the identical random effects structure as the primary (naive) meta-analytic model.

We report results for both, PET and PEESE, with the first reported one being the primary (based on the conditional estimator).

WAAP-WLS The combined WAAP-WLS estimator (weighted average of the adequately powered - weighted least squares) tries to identify studies that are adequately powered to detect the meta-analytic effect. If there is less than two such studies, the method falls back to the WLS estimator (Stanley & Doucouliagos, 2015). If there are at least two adequately powered studies, WAAP returns a WLS estimate based on effects from only those studies.

type = 1: WAAP estimate, 2: WLS estimate. kAdequate = number of adequately powered studies

p-uniform P-uniform* is a selection model conceptually similar to p-curve. It makes use of the fact that p-values follow a uniform distribution at the true effect size while it includes also nonsignificant effect sizes. Permutation-based new version of p-uniform method, the so-called p-uniform* (van Aert, van Assen, 2021).

p-curve Permutation-based p-curve method. Output should be self-explanatory. For more info see p-curve.com

Power for detecting SESOI and bias-corrected parameter estimates Estimates of the statistical power for detecting a smallest effect sizes of interest equal to .20, .50, and .70 in SD units (Cohen’s d). A sort of a thought experiment, we also assumed that population true values equal the bias-corrected estimates (4/3PSM or PET-PEESE) and computed power for those.

Handling of dependencies in bias-correction methods To handle dependencies among the effects, the 4PSM, p-curve, p-uniform are implemented using a permutation-based procedure, randomly selecting only one focal effect (i.e., excluding those which were not coded as being focal) from a single study and iterating nIterations times. Lastly, the procedure selects the result with the median value of the ES estimate (4PSM, p-uniform) or median z-score of the full p-curve (p-curve).

Descriptives

Sample sizes

N of effects

## [1] 192

N of studies

## [1] 189

N of papers

## [1] 76

N of effects eligible just for evidential value test

## [1] 91

Median N across all the ES eligible for meta-analysis

## [1] 69

Meta-analysis results

Directionality of social thermoregulation effects

Source target directionality

## $`Model results`
## $`Model results`$test
##                                          Coef. Estimate     SE t-stat p-val (z)
## 1 factor(sourceTargetDirectionality_reconcil)0    0.465 0.0424  10.97    <0.001
## 2 factor(sourceTargetDirectionality_reconcil)1    0.421 0.0715   5.88    <0.001
##   Sig.
## 1  ***
## 2  ***
## 
## $`Model results`$CIs
##                                           Coef Estimate     SE d.f.
## 1 factor(sourceTargetDirectionality_reconcil)0    0.465 0.0424  Inf
## 2 factor(sourceTargetDirectionality_reconcil)1    0.421 0.0715  Inf
##   Lower 95% CI Upper 95% CI
## 1        0.382        0.548
## 2        0.281        0.561
## 
## 
## $`RVE Wald test`
##  test Fstat df_num df_denom p_val sig
##   HTZ 0.277      1     39.2 0.602

Moderation by prior experiences in relationships

## $`RMA results with model-based SEs`
## 
## Multivariate Meta-Analysis Model (k = 19; method: REML)
## 
## Variance Components:
## 
##             estim    sqrt  nlvls  fixed        factor 
## sigma^2.1  0.0177  0.1329      9     no         study 
## sigma^2.2  0.1215  0.3486     19     no  study/result 
## 
## Test for Heterogeneity:
## Q(df = 18) = 107.2633, p-val < .0001
## 
## Model Results:
## 
## estimate      se    zval    pval   ci.lb   ci.ub 
##   0.5964  0.1208  4.9371  <.0001  0.3596  0.8332  *** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## $`RVE SEs with Satterthwaite small-sample correction`
## $`RVE SEs with Satterthwaite small-sample correction`$test
##     Coef. Estimate   SE t-stat d.f. p-val (Satt) Sig.
## 1 intrcpt    0.596 0.12   4.96 6.46      0.00207   **
## 
## $`RVE SEs with Satterthwaite small-sample correction`$CIs
##      Coef Estimate   SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt    0.596 0.12 6.46        0.307        0.886
## 
## 
## $`Prediction interval`
## 95% PI LB 95% PI UB 
##    -0.307     1.500 
## 
## $Heterogeneity
##                           Tau                           I^2 
##                     0.3731153                    79.3790303 
##                 Jackson's I^2 Between-cluster heterogeneity 
##                    87.9600000                    10.0700000 
##  Within-cluster heterogeneity                           ICC 
##                    69.3000000                     0.1300000 
## 
## $`Proportion of significant results`
## [1] 0.7894737
## 
## $`Publication bias`
## $`Publication bias`$`ES-precision correlation`
## [1] 0.2286703
## 
## $`Publication bias`$`4/3PSM`
##    est     se zvalue pvalue   ciLB   ciUB      k  steps 
##  0.340  0.265  1.284  0.199 -0.179  0.859  9.000  2.000 
## 
## $`Publication bias`$`PET-PEESE`
##   PET estimate             se         zvalue         pvalue           ciLB 
##          0.177          0.431          0.411          0.694         -0.842 
##           ciUB PEESE estimate             se         zvalue         pvalue 
##          1.196          0.378          0.257          1.472          0.185 
##           ciLB           ciUB 
##         -0.229          0.986 
## 
## $`Publication bias`$`WAAP-WLS`
##     method term  estimate std.error statistic     p.value  conf.low conf.high
## 1 WAAP-WLS   b0 0.5006211 0.1378544 0.1378544 0.006670333 0.1827282 0.8185139
##   type kAdequate
## 1    1         9
## 
## $`Publication bias`$`p-uniform*`
##        est       ciLB       ciUB     pvalue 
## 0.51375198 0.07827554 0.89937291 0.12258993 
## 
## $`Publication bias`$`p-curve`
## P-curve analysis 
##  ----------------------- 
## - Total number of provided studies: k = 9 
## - Total number of p<0.05 studies included into the analysis: k = 7 (77.78%) 
## - Total number of studies with p<0.025: k = 6 (66.67%) 
##    
## Results 
##  ----------------------- 
##                     pBinomial  zFull pFull  zHalf pHalf
## Right-skewness test     0.062 -7.249     0 -7.418     0
## Flatness test           0.905  5.068     1  6.907     1
## Note: p-values of 0 or 1 correspond to p<0.001 and p>0.999, respectively.   
## Power Estimate: 98% (90.8%-99%)
##    
## Evidential value 
##  ----------------------- 
## - Evidential value present: yes 
## - Evidential value absent/inadequate: no 
## 
## 
## $`Power for detecting SESOI and bias-corrected parameter estimates`
##              Median power for detecting a SESOI of d = .20 
##                                                      0.227 
##              Median power for detecting a SESOI of d = .50 
##                                                      0.856 
##              Median power for detecting a SESOI of d = .70 
##                                                      0.988 
## Median power for detecting PET-PEESE estimate.PET estimate 
##                                                      0.188 
##             Median power for detecting 4/3PSM estimate.est 
##                                                      0.538

Forest plot and Contour-enhanced funnel plot

P-curve plot

Effect type

## [1] "The compensatory vs priming effects conceptualized by the actual direction of the effect as contrast vs. assimilation"

Compensatory

## $`RMA results with model-based SEs`
## 
## Multivariate Meta-Analysis Model (k = 45; method: REML)
## 
## Variance Components:
## 
##             estim    sqrt  nlvls  fixed        factor 
## sigma^2.1  0.0348  0.1866     35     no         study 
## sigma^2.2  0.0073  0.0855     45     no  study/result 
## 
## Test for Heterogeneity:
## Q(df = 44) = 132.7217, p-val < .0001
## 
## Model Results:
## 
## estimate      se    zval    pval   ci.lb   ci.ub 
##   0.3311  0.0446  7.4258  <.0001  0.2437  0.4185  *** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## $`RVE SEs with Satterthwaite small-sample correction`
## $`RVE SEs with Satterthwaite small-sample correction`$test
##     Coef. Estimate     SE t-stat d.f. p-val (Satt) Sig.
## 1 intrcpt    0.331 0.0446   7.43 30.7       <0.001  ***
## 
## $`RVE SEs with Satterthwaite small-sample correction`$CIs
##      Coef Estimate     SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt    0.331 0.0446 30.7         0.24        0.422
## 
## 
## $`Prediction interval`
## 95% PI LB 95% PI UB 
##    -0.096     0.758 
## 
## $Heterogeneity
##                           Tau                           I^2 
##                     0.2052501                    71.6443532 
##                 Jackson's I^2 Between-cluster heterogeneity 
##                    83.2400000                    59.2000000 
##  Within-cluster heterogeneity                           ICC 
##                    12.4400000                     0.8300000 
## 
## $`Proportion of significant results`
## [1] 0.5106383
## 
## $`Publication bias`
## $`Publication bias`$`ES-precision correlation`
## [1] 0.58586
## 
## $`Publication bias`$`4/3PSM`
##    est     se zvalue pvalue   ciLB   ciUB      k  steps 
##  0.263  0.069  3.841  0.000  0.129  0.397 35.000  2.000 
## 
## $`Publication bias`$`PET-PEESE`
## PEESE estimate             se         zvalue         pvalue           ciLB 
##          0.214          0.067          3.200          0.003          0.078 
##           ciUB   PET estimate             se         zvalue         pvalue 
##          0.351          0.038          0.117          0.326          0.747 
##           ciLB           ciUB 
##         -0.200          0.276 
## 
## $`Publication bias`$`WAAP-WLS`
##     method term  estimate std.error statistic   p.value   conf.low conf.high
## 1 WAAP-WLS   b0 0.2408241 0.1099448 0.1099448 0.1162097 -0.1090692 0.5907174
##   type kAdequate
## 1    1         4
## 
## $`Publication bias`$`p-uniform*`
##          est         ciLB         ciUB       pvalue 
## 0.2776763154 0.1450814742 0.4067088443 0.0005412508 
## 
## $`Publication bias`$`p-curve`
## P-curve analysis 
##  ----------------------- 
## - Total number of provided studies: k = 52 
## - Total number of p<0.05 studies included into the analysis: k = 36 (69.23%) 
## - Total number of studies with p<0.025: k = 25 (48.08%) 
##    
## Results 
##  ----------------------- 
##                     pBinomial  zFull pFull  zHalf pHalf
## Right-skewness test     0.014 -6.934 0.000 -6.997     0
## Flatness test           0.459  3.037 0.999  9.525     1
## Note: p-values of 0 or 1 correspond to p<0.001 and p>0.999, respectively.   
## Power Estimate: 68% (49.9%-81.1%)
##    
## Evidential value 
##  ----------------------- 
## - Evidential value present: yes 
## - Evidential value absent/inadequate: no 
## 
## 
## $`Power for detecting SESOI and bias-corrected parameter estimates`
##                Median power for detecting a SESOI of d = .20 
##                                                        0.385 
##                Median power for detecting a SESOI of d = .50 
##                                                        0.986 
##                Median power for detecting a SESOI of d = .70 
##                                                        1.000 
## Median power for detecting PET-PEESE estimate.PEESE estimate 
##                                                        0.430 
##               Median power for detecting 4/3PSM estimate.est 
##                                                        0.592

Priming

## $`RMA results with model-based SEs`
## 
## Multivariate Meta-Analysis Model (k = 125; method: REML)
## 
## Variance Components:
## 
##             estim    sqrt  nlvls  fixed        factor 
## sigma^2.1  0.0347  0.1864     84     no         study 
## sigma^2.2  0.0196  0.1399    125     no  study/result 
## 
## Test for Heterogeneity:
## Q(df = 124) = 315.2924, p-val < .0001
## 
## Model Results:
## 
## estimate      se     zval    pval   ci.lb   ci.ub 
##   0.4497  0.0339  13.2675  <.0001  0.3833  0.5162  *** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## $`RVE SEs with Satterthwaite small-sample correction`
## $`RVE SEs with Satterthwaite small-sample correction`$test
##     Coef. Estimate     SE t-stat d.f. p-val (Satt) Sig.
## 1 intrcpt     0.45 0.0339   13.3 73.9       <0.001  ***
## 
## $`RVE SEs with Satterthwaite small-sample correction`$CIs
##      Coef Estimate     SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt     0.45 0.0339 73.9        0.382        0.517
## 
## 
## $`Prediction interval`
## 95% PI LB 95% PI UB 
##    -0.019     0.918 
## 
## $Heterogeneity
##                           Tau                           I^2 
##                     0.2330976                    60.9134712 
##                 Jackson's I^2 Between-cluster heterogeneity 
##                    76.8600000                    38.9600000 
##  Within-cluster heterogeneity                           ICC 
##                    21.9600000                     0.6400000 
## 
## $`Proportion of significant results`
## [1] 0.6136364
## 
## $`Publication bias`
## $`Publication bias`$`ES-precision correlation`
## [1] 0.5434097
## 
## $`Publication bias`$`4/3PSM`
##    est     se zvalue pvalue   ciLB   ciUB      k  steps 
##  0.248  0.052  4.780  0.000  0.147  0.350 83.000  2.000 
## 
## $`Publication bias`$`PET-PEESE`
## PEESE estimate             se         zvalue         pvalue           ciLB 
##          0.204          0.056          3.656          0.000          0.093 
##           ciUB   PET estimate             se         zvalue         pvalue 
##          0.315         -0.049          0.060         -0.810          0.421 
##           ciLB           ciUB 
##         -0.169          0.071 
## 
## $`Publication bias`$`WAAP-WLS`
##     method term   estimate  std.error  statistic   p.value    conf.low
## 1 WAAP-WLS   b0 0.06033676 0.03493944 0.03493944 0.1349326 -0.02515696
##   conf.high type kAdequate
## 1 0.1458305    1         7
## 
## $`Publication bias`$`p-uniform*`
##           est          ciLB          ciUB        pvalue 
## 0.28237593329 0.17999288439 0.38133351331 0.00001748352 
## 
## $`Publication bias`$`p-curve`
## P-curve analysis 
##  ----------------------- 
## - Total number of provided studies: k = 116 
## - Total number of p<0.05 studies included into the analysis: k = 80 (68.97%) 
## - Total number of studies with p<0.025: k = 50 (43.1%) 
##    
## Results 
##  ----------------------- 
##                     pBinomial  zFull pFull  zHalf pHalf
## Right-skewness test     0.016 -6.251 0.000 -8.037     0
## Flatness test           0.053  0.347 0.636  9.925     1
## Note: p-values of 0 or 1 correspond to p<0.001 and p>0.999, respectively.   
## Power Estimate: 36% (23.4%-49.7%)
##    
## Evidential value 
##  ----------------------- 
## - Evidential value present: yes 
## - Evidential value absent/inadequate: no 
## 
## 
## $`Power for detecting SESOI and bias-corrected parameter estimates`
##                Median power for detecting a SESOI of d = .20 
##                                                        0.202 
##                Median power for detecting a SESOI of d = .50 
##                                                        0.801 
##                Median power for detecting a SESOI of d = .70 
##                                                        0.976 
## Median power for detecting PET-PEESE estimate.PEESE estimate 
##                                                        0.209 
##               Median power for detecting 4/3PSM estimate.est 
##                                                        0.286

Table 1

##              k   g [95% CI]        SE   Tau  I^2 3PSM est [95% CI] 3PSM.pvalue
## Compensatory 47  0.33 [0.24, 0.42] 0.04 0.21 72% 0.26 [0.13, 0.4]  0          
## Priming      132 0.45 [0.38, 0.52] 0.03 0.23 61% 0.25 [0.15, 0.35] 0          
##              PET-PEESE est [95% CI] PET-PEESE.pvalue
## Compensatory 0.21 [0.08, 0.35]      0.003           
## Priming      0.2 [0.09, 0.32]       0

Comparison of effect types

Model without covariates

## $`Model results`
## $`Model results`$test
##                        Coef. Estimate     SE t-stat p-val (z) Sig.
## 1 factor(effectCompPriming)1    0.345 0.0461   7.48    <0.001  ***
## 2 factor(effectCompPriming)2    0.443 0.0324  13.65    <0.001  ***
## 
## $`Model results`$CIs
##                         Coef Estimate     SE d.f. Lower 95% CI Upper 95% CI
## 1 factor(effectCompPriming)1    0.345 0.0461  Inf        0.254        0.435
## 2 factor(effectCompPriming)2    0.443 0.0324  Inf        0.379        0.506
## 
## 
## $`RVE Wald test`
##  test Fstat df_num df_denom p_val sig
##   HTZ  2.81      1     47.5   0.1

Model with covariates

Controlling for design-related factors that are prognostic w.r.t. the effect sizes (i.e., might vary across moderator categories), namely rct, published, sourceTargetDirectionality, and studentSample.

## $`Model results`
## $`Model results`$test
##                                 Coef. Estimate     SE t-stat p-val (z) Sig.
## 1          factor(effectCompPriming)1   0.0932 0.0721  1.293   0.19600     
## 2          factor(effectCompPriming)2   0.1769 0.0898  1.969   0.04896    *
## 3                                 rct   0.1463 0.0564  2.595   0.00945   **
## 4                           published   0.1285 0.0630  2.040   0.04138    *
## 5 sourceTargetDirectionality_reconcil  -0.0305 0.0681 -0.448   0.65428     
## 6                       studentSample   0.0789 0.0684  1.153   0.24873     
## 
## $`Model results`$CIs
##                                  Coef Estimate     SE d.f. Lower 95% CI
## 1          factor(effectCompPriming)1   0.0932 0.0721  Inf    -0.048070
## 2          factor(effectCompPriming)2   0.1769 0.0898  Inf     0.000805
## 3                                 rct   0.1463 0.0564  Inf     0.035802
## 4                           published   0.1285 0.0630  Inf     0.005026
## 5 sourceTargetDirectionality_reconcil  -0.0305 0.0681  Inf    -0.163877
## 6                       studentSample   0.0789 0.0684  Inf    -0.055188
##   Upper 95% CI
## 1        0.234
## 2        0.353
## 3        0.257
## 4        0.252
## 5        0.103
## 6        0.213
## 
## 
## $`RVE Wald test`
##  test Fstat df_num df_denom p_val sig
##   HTZ  2.13      1     26.1 0.156

Plots

Forest plots

Contour-enhanced funnel plots

p-curve plots

PET-PEESE plots

top = Compensatory, bottom = Priming

Using the sqrt(2/n) and 2/n terms instead of SE and var for PET and PEESE, respectively since modified sample-size based estimator was implemented (see https://www.jepusto.com/pet-peese-performance/).

Mood

Results

## $`RMA results with model-based SEs`
## 
## Multivariate Meta-Analysis Model (k = 17; method: REML)
## 
## Variance Components:
## 
##             estim    sqrt  nlvls  fixed        factor 
## sigma^2.1  0.0000  0.0000     11     no         study 
## sigma^2.2  0.0000  0.0000     17     no  study/result 
## 
## Test for Heterogeneity:
## Q(df = 16) = 7.1492, p-val = 0.9703
## 
## Model Results:
## 
## estimate      se    zval    pval   ci.lb   ci.ub 
##   0.2315  0.0698  3.3145  0.0009  0.0946  0.3684  *** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## $`RVE SEs with Satterthwaite small-sample correction`
## $`RVE SEs with Satterthwaite small-sample correction`$test
##     Coef. Estimate     SE t-stat d.f. p-val (Satt) Sig.
## 1 intrcpt    0.231 0.0448   5.17  8.9       <0.001  ***
## 
## $`RVE SEs with Satterthwaite small-sample correction`$CIs
##      Coef Estimate     SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt    0.231 0.0448  8.9         0.13        0.333
## 
## 
## $`Prediction interval`
## 95% PI LB 95% PI UB 
##     0.131     0.332 
## 
## $Heterogeneity
##                           Tau                           I^2 
##              0.00000295647900              0.00000001354476 
##                 Jackson's I^2 Between-cluster heterogeneity 
##              0.00000000000000              0.00000000000000 
##  Within-cluster heterogeneity                           ICC 
##              0.00000000000000              1.00000000000000 
## 
## $`Proportion of significant results`
## [1] 0.1176471
## 
## $`Publication bias`
## [1] "Publication bias corrections not carried out"
## 
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## [1] "Power for detecting bias-corrected parameter estimates not computed"

Forest plot and Contour-enhanced funnel plot

Overall effect results

Results

Number of iterations run equal to 200 for p-curve and 5000 for all other bias correction functions.

## $`RMA results with model-based SEs`
## 
## Multivariate Meta-Analysis Model (k = 175; method: REML)
## 
## Variance Components:
## 
##             estim    sqrt  nlvls  fixed        factor 
## sigma^2.1  0.0438  0.2092    116     no         study 
## sigma^2.2  0.0357  0.1888    175     no  study/result 
## 
## Test for Heterogeneity:
## Q(df = 174) = 677.7377, p-val < .0001
## 
## Model Results:
## 
## estimate      se     zval    pval   ci.lb   ci.ub 
##   0.4085  0.0315  12.9864  <.0001  0.3469  0.4702  *** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## $`RVE SEs with Satterthwaite small-sample correction`
## $`RVE SEs with Satterthwaite small-sample correction`$test
##     Coef. Estimate     SE t-stat d.f. p-val (Satt) Sig.
## 1 intrcpt    0.409 0.0315     13  105       <0.001  ***
## 
## $`RVE SEs with Satterthwaite small-sample correction`$CIs
##      Coef Estimate     SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt    0.409 0.0315  105        0.346        0.471
## 
## 
## $`Prediction interval`
## 95% PI LB 95% PI UB 
##    -0.153     0.970 
## 
## $Heterogeneity
##                           Tau                           I^2 
##                     0.2818135                    74.9082576 
##                 Jackson's I^2 Between-cluster heterogeneity 
##                    85.1400000                    41.2700000 
##  Within-cluster heterogeneity                           ICC 
##                    33.6400000                     0.5500000 
## 
## $`Proportion of significant results`
## [1] 0.5923913
## 
## $`Publication bias`
## $`Publication bias`$`ES-precision correlation`
## [1] 0.5796001
## 
## $`Publication bias`$`4/3PSM`
##     est      se  zvalue  pvalue    ciLB    ciUB       k   steps 
##   0.187   0.052   3.599   0.000   0.085   0.289 114.000   2.000 
## 
## $`Publication bias`$`PET-PEESE`
## PEESE estimate             se         zvalue         pvalue           ciLB 
##          0.159          0.053          3.008          0.003          0.054 
##           ciUB   PET estimate             se         zvalue         pvalue 
##          0.264         -0.138          0.072         -1.915          0.058 
##           ciLB           ciUB 
##         -0.280          0.005 
## 
## $`Publication bias`$`WAAP-WLS`
##     method term  estimate  std.error  statistic   p.value   conf.low conf.high
## 1 WAAP-WLS   b0 0.1493071 0.09129957 0.09129957 0.2436033 -0.2435232 0.5421375
##   type kAdequate
## 1    1         3
## 
## $`Publication bias`$`p-uniform*`
##          est         ciLB         ciUB       pvalue 
## 0.1944982257 0.0957078307 0.2920326952 0.0005655032 
## 
## $`Publication bias`$`p-curve`
## P-curve analysis 
##  ----------------------- 
## - Total number of provided studies: k = 166 
## - Total number of p<0.05 studies included into the analysis: k = 115 (69.28%) 
## - Total number of studies with p<0.025: k = 75 (45.18%) 
##    
## Results 
##  ----------------------- 
##                     pBinomial  zFull pFull   zHalf pHalf
## Right-skewness test     0.001 -8.782 0.000 -10.217     0
## Flatness test           0.087  1.653 0.951  13.089     1
## Note: p-values of 0 or 1 correspond to p<0.001 and p>0.999, respectively.   
## Power Estimate: 45% (33.4%-55.5%)
##    
## Evidential value 
##  ----------------------- 
## - Evidential value present: yes 
## - Evidential value absent/inadequate: no 
## 
## 
## $`Power for detecting SESOI and bias-corrected parameter estimates`
##                Median power for detecting a SESOI of d = .20 
##                                                        0.215 
##                Median power for detecting a SESOI of d = .50 
##                                                        0.830 
##                Median power for detecting a SESOI of d = .70 
##                                                        0.983 
## Median power for detecting PET-PEESE estimate.PEESE estimate 
##                                                        0.153 
##               Median power for detecting 4/3PSM estimate.est 
##                                                        0.193

Forest plot

Contour-enhanced funnel plot

p-curve plot

Methods

Moderator analysis

## $`Model results`
## $`Model results`$test
##                                     Coef. Estimate     SE t-stat p-val (z) Sig.
## 1 methodPhysical temperature manipulation    0.464 0.0629   7.37   < 0.001  ***
## 2   methodVisual/verbal temperature prime    0.485 0.0589   8.23   < 0.001  ***
## 3               methodOutside temperature    0.379 0.1512   2.51   0.01223    *
## 4        methodTemperature estimate as DV    0.477 0.0703   6.79   < 0.001  ***
## 5  methodSubjective warmth judgment as DV    0.299 0.1149   2.61   0.00915   **
## 
## $`Model results`$CIs
##                                      Coef Estimate     SE d.f. Lower 95% CI
## 1 methodPhysical temperature manipulation    0.464 0.0629  Inf       0.3406
## 2   methodVisual/verbal temperature prime    0.485 0.0589  Inf       0.3693
## 3               methodOutside temperature    0.379 0.1512  Inf       0.0825
## 4        methodTemperature estimate as DV    0.477 0.0703  Inf       0.3396
## 5  methodSubjective warmth judgment as DV    0.299 0.1149  Inf       0.0743
##   Upper 95% CI
## 1        0.587
## 2        0.600
## 3        0.675
## 4        0.615
## 5        0.525
## 
## 
## $`RVE Wald test`
##  test Fstat df_num df_denom p_val sig
##   HTZ 0.682      4     17.5 0.614

Subgroup analysis

Leaving out the Core temperature measurement and Skin temperature measurement, since k is too low.

Meta-analysis results

Brief results

##                                   k  g [95% CI]         SE   Tau  I^2
## Physical.temperature.manipulation 83 0.48 [0.36, 0.59]  0.06 0.34 69%
## Visual.verbal.temperature.prime   23 0.44 [0.35, 0.53]  0.04 0.15 38%
## Outside.temperature               13 0.16 [-0.02, 0.34] 0.06 0.1  44%
## Temperature.estimate.as.DV        25 0.39 [0.27, 0.52]  0.06 0.19 60%
## Subjective.warmth.judgment.as.DV  14 0.28 [0.03, 0.53]  0.11 0.37 93%

Physical temperature manipulation

## $`RMA results with model-based SEs`
## 
## Multivariate Meta-Analysis Model (k = 82; method: REML)
## 
## Variance Components:
## 
##             estim    sqrt  nlvls  fixed        factor 
## sigma^2.1  0.1003  0.3167     51     no         study 
## sigma^2.2  0.0153  0.1238     82     no  study/result 
## 
## Test for Heterogeneity:
## Q(df = 81) = 339.5264, p-val < .0001
## 
## Model Results:
## 
## estimate      se    zval    pval   ci.lb   ci.ub 
##   0.4779  0.0575  8.3075  <.0001  0.3652  0.5907  *** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## $`RVE SEs with Satterthwaite small-sample correction`
## $`RVE SEs with Satterthwaite small-sample correction`$test
##     Coef. Estimate     SE t-stat d.f. p-val (Satt) Sig.
## 1 intrcpt    0.478 0.0576    8.3 48.2       <0.001  ***
## 
## $`RVE SEs with Satterthwaite small-sample correction`$CIs
##      Coef Estimate     SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt    0.478 0.0576 48.2        0.362        0.594
## 
## 
## $`Prediction interval`
## 95% PI LB 95% PI UB 
##    -0.215     1.171 
## 
## $Heterogeneity
##                           Tau                           I^2 
##                     0.3400426                    69.0555528 
##                 Jackson's I^2 Between-cluster heterogeneity 
##                    80.9300000                    59.9000000 
##  Within-cluster heterogeneity                           ICC 
##                     9.1500000                     0.8700000 
## 
## $`Proportion of significant results`
## [1] 0.6746988
## 
## $`Publication bias`
## [1] "Publication bias corrections not carried out"
## 
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## [1] "Power for detecting bias-corrected parameter estimates not computed"

Visual/verbal temperature prime

## $`RMA results with model-based SEs`
## 
## Multivariate Meta-Analysis Model (k = 23; method: REML)
## 
## Variance Components:
## 
##             estim    sqrt  nlvls  fixed        factor 
## sigma^2.1  0.0000  0.0000     14     no         study 
## sigma^2.2  0.0232  0.1524     23     no  study/result 
## 
## Test for Heterogeneity:
## Q(df = 22) = 38.5260, p-val = 0.0160
## 
## Model Results:
## 
## estimate      se    zval    pval   ci.lb   ci.ub 
##   0.4398  0.0600  7.3355  <.0001  0.3223  0.5573  *** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## $`RVE SEs with Satterthwaite small-sample correction`
## $`RVE SEs with Satterthwaite small-sample correction`$test
##     Coef. Estimate     SE t-stat d.f. p-val (Satt) Sig.
## 1 intrcpt     0.44 0.0411   10.7 11.4       <0.001  ***
## 
## $`RVE SEs with Satterthwaite small-sample correction`$CIs
##      Coef Estimate     SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt     0.44 0.0411 11.4         0.35         0.53
## 
## 
## $`Prediction interval`
## 95% PI LB 95% PI UB 
##     0.099     0.781 
## 
## $Heterogeneity
##                           Tau                           I^2 
##                     0.1523991                    37.8013636 
##                 Jackson's I^2 Between-cluster heterogeneity 
##                    13.6800000                     0.0000000 
##  Within-cluster heterogeneity                           ICC 
##                    37.8000000                     0.0000000 
## 
## $`Proportion of significant results`
## [1] 0.6956522
## 
## $`Publication bias`
## [1] "Publication bias corrections not carried out"
## 
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## [1] "Power for detecting bias-corrected parameter estimates not computed"

Outside temperature

## $`RMA results with model-based SEs`
## 
## Multivariate Meta-Analysis Model (k = 8; method: REML)
## 
## Variance Components:
## 
##             estim    sqrt  nlvls  fixed        factor 
## sigma^2.1  0.0077  0.0875      6     no         study 
## sigma^2.2  0.0028  0.0525      8     no  study/result 
## 
## Test for Heterogeneity:
## Q(df = 7) = 14.2050, p-val = 0.0477
## 
## Model Results:
## 
## estimate      se    zval    pval   ci.lb   ci.ub 
##   0.1607  0.0642  2.5034  0.0123  0.0349  0.2864  * 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## $`RVE SEs with Satterthwaite small-sample correction`
## $`RVE SEs with Satterthwaite small-sample correction`$test
##     Coef. Estimate     SE t-stat d.f. p-val (Satt) Sig.
## 1 intrcpt    0.161 0.0634   2.54 3.63       0.0705    .
## 
## $`RVE SEs with Satterthwaite small-sample correction`$CIs
##      Coef Estimate     SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt    0.161 0.0634 3.63      -0.0225        0.344
## 
## 
## $`Prediction interval`
## 95% PI LB 95% PI UB 
##    -0.147     0.468 
## 
## $Heterogeneity
##                           Tau                           I^2 
##                     0.1020439                    44.1844740 
##                 Jackson's I^2 Between-cluster heterogeneity 
##                    56.9100000                    32.5100000 
##  Within-cluster heterogeneity                           ICC 
##                    11.6700000                     0.7400000 
## 
## $`Proportion of significant results`
## [1] 0.1538462
## 
## $`Publication bias`
## [1] "Publication bias corrections not carried out"
## 
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## [1] "Power for detecting bias-corrected parameter estimates not computed"

Temperature estimate as DV

## $`RMA results with model-based SEs`
## 
## Multivariate Meta-Analysis Model (k = 23; method: REML)
## 
## Variance Components:
## 
##             estim    sqrt  nlvls  fixed        factor 
## sigma^2.1  0.0378  0.1945     21     no         study 
## sigma^2.2  0.0000  0.0000     23     no  study/result 
## 
## Test for Heterogeneity:
## Q(df = 22) = 49.5679, p-val = 0.0007
## 
## Model Results:
## 
## estimate      se    zval    pval   ci.lb   ci.ub 
##   0.3931  0.0599  6.5631  <.0001  0.2757  0.5104  *** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## $`RVE SEs with Satterthwaite small-sample correction`
## $`RVE SEs with Satterthwaite small-sample correction`$test
##     Coef. Estimate     SE t-stat d.f. p-val (Satt) Sig.
## 1 intrcpt    0.393 0.0601   6.54   17       <0.001  ***
## 
## $`RVE SEs with Satterthwaite small-sample correction`$CIs
##      Coef Estimate     SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt    0.393 0.0601   17        0.266         0.52
## 
## 
## $`Prediction interval`
## 95% PI LB 95% PI UB 
##    -0.031     0.817 
## 
## $Heterogeneity
##                           Tau                           I^2 
##                     0.1945094                    59.9035594 
##                 Jackson's I^2 Between-cluster heterogeneity 
##                    71.0500000                    59.9000000 
##  Within-cluster heterogeneity                           ICC 
##                     0.0000000                     1.0000000 
## 
## $`Proportion of significant results`
## [1] 0.72
## 
## $`Publication bias`
## [1] "Publication bias corrections not carried out"
## 
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## [1] "Power for detecting bias-corrected parameter estimates not computed"

Subjective warmth judgment as DV

## $`RMA results with model-based SEs`
## 
## Multivariate Meta-Analysis Model (k = 13; method: REML)
## 
## Variance Components:
## 
##             estim    sqrt  nlvls  fixed        factor 
## sigma^2.1  0.1253  0.3539     12     no         study 
## sigma^2.2  0.0089  0.0942     13     no  study/result 
## 
## Test for Heterogeneity:
## Q(df = 12) = 69.2488, p-val < .0001
## 
## Model Results:
## 
## estimate      se    zval    pval   ci.lb   ci.ub 
##   0.2810  0.1153  2.4362  0.0148  0.0549  0.5070  * 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## $`RVE SEs with Satterthwaite small-sample correction`
## $`RVE SEs with Satterthwaite small-sample correction`$test
##     Coef. Estimate    SE t-stat d.f. p-val (Satt) Sig.
## 1 intrcpt    0.281 0.115   2.45 10.7        0.033    *
## 
## $`RVE SEs with Satterthwaite small-sample correction`$CIs
##      Coef Estimate    SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt    0.281 0.115 10.7       0.0274        0.535
## 
## 
## $`Prediction interval`
## 95% PI LB 95% PI UB 
##    -0.564     1.126 
## 
## $Heterogeneity
##                           Tau                           I^2 
##                     0.3662546                    92.6858591 
##                 Jackson's I^2 Between-cluster heterogeneity 
##                    95.7600000                    86.5600000 
##  Within-cluster heterogeneity                           ICC 
##                     6.1300000                     0.9300000 
## 
## $`Proportion of significant results`
## [1] 0.5714286
## 
## $`Publication bias`
## [1] "Publication bias corrections not carried out"
## 
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## [1] "Power for detecting bias-corrected parameter estimates not computed"

Bias correction results

Physical temperature manipulation

## $`Publication bias`
## $`Publication bias`$`ES-precision correlation`
## [1] 0.5226101
## 
## $`Publication bias`$`4/3PSM`
##    est     se zvalue pvalue   ciLB   ciUB      k  steps 
##  0.151  0.094  1.603  0.109 -0.034  0.336 51.000  2.000 
## 
## $`Publication bias`$`PET-PEESE`
##   PET estimate             se         zvalue         pvalue           ciLB 
##         -0.254          0.225         -1.128          0.265         -0.705 
##           ciUB PEESE estimate             se         zvalue         pvalue 
##          0.198          0.190          0.133          1.424          0.161 
##           ciLB           ciUB 
##         -0.078          0.457 
## 
## $`Publication bias`$`WAAP-WLS`
##     method term   estimate std.error statistic   p.value  conf.low conf.high
## 1 WAAP-WLS   b0 -0.4364618  0.251926  0.251926 0.2253231 -1.520412 0.6474885
##   type kAdequate
## 1    1         3
## 
## $`Publication bias`$`p-uniform*`
##         est        ciLB        ciUB      pvalue 
##  0.06957033 -0.13193200  0.27026087  0.52185334 
## 
## $`Publication bias`$`p-curve`
## P-curve analysis 
##  ----------------------- 
## - Total number of provided studies: k = 82 
## - Total number of p<0.05 studies included into the analysis: k = 58 (70.73%) 
## - Total number of studies with p<0.025: k = 33 (40.24%) 
##    
## Results 
##  ----------------------- 
##                     pBinomial  zFull pFull  zHalf pHalf
## Right-skewness test     0.179 -6.720 0.000 -8.942     0
## Flatness test           0.013  1.791 0.963 10.255     1
## Note: p-values of 0 or 1 correspond to p<0.001 and p>0.999, respectively.   
## Power Estimate: 50% (34.7%-65%)
##    
## Evidential value 
##  ----------------------- 
## - Evidential value present: yes 
## - Evidential value absent/inadequate: no 
## 
## 
## $`Power for detecting SESOI and bias-corrected parameter estimates`
##              Median power for detecting a SESOI of d = .20 
##                                                     "0.19" 
##              Median power for detecting a SESOI of d = .50 
##                                                    "0.768" 
##              Median power for detecting a SESOI of d = .70 
##                                                    "0.965" 
## Median power for detecting PET-PEESE estimate.PET estimate 
##                    "ES estimate in the opposite direction" 
##             Median power for detecting 4/3PSM estimate.est 
##                                                    "0.129"

Visual/verbal temperature prime

## $`Publication bias`
## $`Publication bias`$`ES-precision correlation`
## [1] 0.3287924
## 
## $`Publication bias`$`4/3PSM`
##    est     se zvalue pvalue   ciLB   ciUB      k  steps 
##  0.271  0.080  3.400  0.001  0.115  0.428 14.000  2.000 
## 
## $`Publication bias`$`PET-PEESE`
## PEESE estimate             se         zvalue         pvalue           ciLB 
##          0.292          0.110          2.662          0.021          0.053 
##           ciUB   PET estimate             se         zvalue         pvalue 
##          0.532          0.182          0.207          0.878          0.397 
##           ciLB           ciUB 
##         -0.270          0.634 
## 
## $`Publication bias`$`WAAP-WLS`
##     method term  estimate  std.error  statistic     p.value  conf.low conf.high
## 1 WAAP-WLS   b0 0.3037822 0.03003057 0.03003057 0.009631472 0.1745711 0.4329934
##   type kAdequate
## 1    1         3
## 
## $`Publication bias`$`p-uniform*`
##        est       ciLB       ciUB     pvalue 
## 0.31186560 0.09457384 0.49685638 0.23877258 
## 
## $`Publication bias`$`p-curve`
## P-curve analysis 
##  ----------------------- 
## - Total number of provided studies: k = 23 
## - Total number of p<0.05 studies included into the analysis: k = 19 (82.61%) 
## - Total number of studies with p<0.025: k = 14 (60.87%) 
##    
## Results 
##  ----------------------- 
##                     pBinomial  zFull pFull  zHalf pHalf
## Right-skewness test     0.032 -4.527 0.000 -4.793     0
## Flatness test           0.670  1.450 0.926  5.666     1
## Note: p-values of 0 or 1 correspond to p<0.001 and p>0.999, respectively.   
## Power Estimate: 56% (30.3%-77.3%)
##    
## Evidential value 
##  ----------------------- 
## - Evidential value present: yes 
## - Evidential value absent/inadequate: no 
## 
## 
## $`Power for detecting SESOI and bias-corrected parameter estimates`
##                Median power for detecting a SESOI of d = .20 
##                                                        0.202 
##                Median power for detecting a SESOI of d = .50 
##                                                        0.801 
##                Median power for detecting a SESOI of d = .70 
##                                                        0.976 
## Median power for detecting PET-PEESE estimate.PEESE estimate 
##                                                        0.374 
##               Median power for detecting 4/3PSM estimate.est 
##                                                        0.331

Temperature estimate as DV

## $`Publication bias`
## $`Publication bias`$`ES-precision correlation`
## [1] 0.7500059
## 
## $`Publication bias`$`4/3PSM`
##    est     se zvalue pvalue   ciLB   ciUB      k  steps 
##  0.046  0.058  0.779  0.436 -0.069  0.160 21.000  2.000 
## 
## $`Publication bias`$`PET-PEESE`
##   PET estimate             se         zvalue         pvalue           ciLB 
##         -0.086          0.091         -0.942          0.358         -0.277 
##           ciUB PEESE estimate             se         zvalue         pvalue 
##          0.105          0.108          0.062          1.742          0.098 
##           ciLB           ciUB 
##         -0.022          0.238 
## 
## $`Publication bias`$`WAAP-WLS`
##     method term   estimate std.error statistic   p.value  conf.low conf.high
## 1 WAAP-WLS   b0 0.04964282 0.1242567 0.1242567 0.7580266 -1.529189  1.628474
##   type kAdequate
## 1    1         2
## 
## $`Publication bias`$`p-uniform*`
##        est       ciLB       ciUB     pvalue 
##  0.0556670 -0.1507642  0.2412312  0.6814799 
## 
## $`Publication bias`$`p-curve`
## P-curve analysis 
##  ----------------------- 
## - Total number of provided studies: k = 24 
## - Total number of p<0.05 studies included into the analysis: k = 18 (75%) 
## - Total number of studies with p<0.025: k = 10 (41.67%) 
##    
## Results 
##  ----------------------- 
##                     pBinomial  zFull pFull  zHalf pHalf
## Right-skewness test     0.407 -0.951 0.171 -1.939 0.026
## Flatness test           0.112 -1.590 0.056  3.610 1.000
## Note: p-values of 0 or 1 correspond to p<0.001 and p>0.999, respectively.   
## Power Estimate: 10% (5%-34.4%)
##    
## Evidential value 
##  ----------------------- 
## - Evidential value present: yes 
## - Evidential value absent/inadequate: no 
## 
## 
## $`Power for detecting SESOI and bias-corrected parameter estimates`
##              Median power for detecting a SESOI of d = .20 
##                                                    "0.202" 
##              Median power for detecting a SESOI of d = .50 
##                                                    "0.801" 
##              Median power for detecting a SESOI of d = .70 
##                                                    "0.976" 
## Median power for detecting PET-PEESE estimate.PET estimate 
##                    "ES estimate in the opposite direction" 
##             Median power for detecting 4/3PSM estimate.est 
##                                                    "0.058"

Contour-enhanced funnel plots

Forest plots

Categories

Leaving out the Robotics and Neural Mechanisms, since k is too low

Results for different categories

Meta-analysis results

Brief results

##                          k  g [95% CI]        SE   Tau  I^2
## Emotion                  24 0.39 [0.31, 0.46] 0.03 0.15 54%
## Interpersonal            76 0.36 [0.26, 0.46] 0.05 0.31 78%
## Person.perception        39 0.41 [0.23, 0.58] 0.08 0.33 81%
## Group.processes          12 0.62 [0.38, 0.85] 0.09 0    0% 
## Moral.judgment           6  0.49 [-0.12, 1.1] 0.11 0.03 2% 
## Self.regulation          26 0.32 [0.17, 0.47] 0.07 0.27 76%
## Cognitive.processes      36 0.56 [0.46, 0.66] 0.05 0.12 20%
## Economic.decision.making 43 0.44 [0.28, 0.59] 0.07 0.31 70%

Emotion

## $`RMA results with model-based SEs`
## 
## Multivariate Meta-Analysis Model (k = 23; method: REML)
## 
## Variance Components:
## 
##             estim    sqrt  nlvls  fixed        factor 
## sigma^2.1  0.0000  0.0000     19     no         study 
## sigma^2.2  0.0233  0.1526     23     no  study/result 
## 
## Test for Heterogeneity:
## Q(df = 22) = 44.6519, p-val = 0.0029
## 
## Model Results:
## 
## estimate      se    zval    pval   ci.lb   ci.ub 
##   0.3859  0.0506  7.6308  <.0001  0.2868  0.4850  *** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## $`RVE SEs with Satterthwaite small-sample correction`
## $`RVE SEs with Satterthwaite small-sample correction`$test
##     Coef. Estimate     SE t-stat d.f. p-val (Satt) Sig.
## 1 intrcpt    0.386 0.0343   11.2 15.3       <0.001  ***
## 
## $`RVE SEs with Satterthwaite small-sample correction`$CIs
##      Coef Estimate     SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt    0.386 0.0343 15.3        0.313        0.459
## 
## 
## $`Prediction interval`
## 95% PI LB 95% PI UB 
##     0.057     0.715 
## 
## $Heterogeneity
##                           Tau                           I^2 
##                     0.1526369                    54.0804386 
##                 Jackson's I^2 Between-cluster heterogeneity 
##                    65.7500000                     0.0000000 
##  Within-cluster heterogeneity                           ICC 
##                    54.0800000                     0.0000000 
## 
## $`Proportion of significant results`
## [1] 0.7083333
## 
## $`Publication bias`
## [1] "Publication bias corrections not carried out"
## 
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## [1] "Power for detecting bias-corrected parameter estimates not computed"

Interpersonal

## $`RMA results with model-based SEs`
## 
## Multivariate Meta-Analysis Model (k = 75; method: REML)
## 
## Variance Components:
## 
##             estim    sqrt  nlvls  fixed        factor 
## sigma^2.1  0.0503  0.2242     56     no         study 
## sigma^2.2  0.0488  0.2208     75     no  study/result 
## 
## Test for Heterogeneity:
## Q(df = 74) = 364.3079, p-val < .0001
## 
## Model Results:
## 
## estimate      se    zval    pval   ci.lb   ci.ub 
##   0.3594  0.0498  7.2141  <.0001  0.2617  0.4570  *** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## $`RVE SEs with Satterthwaite small-sample correction`
## $`RVE SEs with Satterthwaite small-sample correction`$test
##     Coef. Estimate     SE t-stat d.f. p-val (Satt) Sig.
## 1 intrcpt    0.359 0.0499   7.21 51.7       <0.001  ***
## 
## $`RVE SEs with Satterthwaite small-sample correction`$CIs
##      Coef Estimate     SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt    0.359 0.0499 51.7        0.259        0.459
## 
## 
## $`Prediction interval`
## 95% PI LB 95% PI UB 
##    -0.279     0.998 
## 
## $Heterogeneity
##                           Tau                           I^2 
##                     0.3146483                    77.6948925 
##                 Jackson's I^2 Between-cluster heterogeneity 
##                    85.2200000                    39.4400000 
##  Within-cluster heterogeneity                           ICC 
##                    38.2600000                     0.5100000 
## 
## $`Proportion of significant results`
## [1] 0.5789474
## 
## $`Publication bias`
## [1] "Publication bias corrections not carried out"
## 
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## [1] "Power for detecting bias-corrected parameter estimates not computed"

Person perception

## $`RMA results with model-based SEs`
## 
## Multivariate Meta-Analysis Model (k = 36; method: REML)
## 
## Variance Components:
## 
##             estim    sqrt  nlvls  fixed        factor 
## sigma^2.1  0.0765  0.2766     21     no         study 
## sigma^2.2  0.0337  0.1835     36     no  study/result 
## 
## Test for Heterogeneity:
## Q(df = 35) = 133.8868, p-val < .0001
## 
## Model Results:
## 
## estimate      se    zval    pval   ci.lb   ci.ub 
##   0.4075  0.0829  4.9175  <.0001  0.2451  0.5698  *** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## $`RVE SEs with Satterthwaite small-sample correction`
## $`RVE SEs with Satterthwaite small-sample correction`$test
##     Coef. Estimate     SE t-stat d.f. p-val (Satt) Sig.
## 1 intrcpt    0.407 0.0828   4.92 18.3       <0.001  ***
## 
## $`RVE SEs with Satterthwaite small-sample correction`$CIs
##      Coef Estimate     SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt    0.407 0.0828 18.3        0.234        0.581
## 
## 
## $`Prediction interval`
## 95% PI LB 95% PI UB 
##    -0.306     1.121 
## 
## $Heterogeneity
##                           Tau                           I^2 
##                     0.3319022                    81.0462124 
##                 Jackson's I^2 Between-cluster heterogeneity 
##                    90.0500000                    56.2800000 
##  Within-cluster heterogeneity                           ICC 
##                    24.7700000                     0.6900000 
## 
## $`Proportion of significant results`
## [1] 0.4358974
## 
## $`Publication bias`
## [1] "Publication bias corrections not carried out"
## 
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## [1] "Power for detecting bias-corrected parameter estimates not computed"

Group processes

## $`RMA results with model-based SEs`
## 
## Multivariate Meta-Analysis Model (k = 11; method: REML)
## 
## Variance Components:
## 
##             estim    sqrt  nlvls  fixed        factor 
## sigma^2.1  0.0000  0.0000      7     no         study 
## sigma^2.2  0.0000  0.0000     11     no  study/result 
## 
## Test for Heterogeneity:
## Q(df = 10) = 8.8815, p-val = 0.5434
## 
## Model Results:
## 
## estimate      se    zval    pval   ci.lb   ci.ub 
##   0.6152  0.1079  5.7024  <.0001  0.4037  0.8266  *** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## $`RVE SEs with Satterthwaite small-sample correction`
## $`RVE SEs with Satterthwaite small-sample correction`$test
##     Coef. Estimate    SE t-stat d.f. p-val (Satt) Sig.
## 1 intrcpt    0.615 0.094   6.55  5.3       <0.001  ***
## 
## $`RVE SEs with Satterthwaite small-sample correction`$CIs
##      Coef Estimate    SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt    0.615 0.094  5.3        0.378        0.853
## 
## 
## $`Prediction interval`
## 95% PI LB 95% PI UB 
##     0.386     0.845 
## 
## $Heterogeneity
##                           Tau                           I^2 
##              0.00000462993551              0.00000002010181 
##                 Jackson's I^2 Between-cluster heterogeneity 
##              0.00000000000000              0.00000000000000 
##  Within-cluster heterogeneity                           ICC 
##              0.00000000000000              0.83000000000000 
## 
## $`Proportion of significant results`
## [1] 0.8333333
## 
## $`Publication bias`
## [1] "Publication bias corrections not carried out"
## 
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## [1] "Power for detecting bias-corrected parameter estimates not computed"

Moral judgment

## $`RMA results with model-based SEs`
## 
## Multivariate Meta-Analysis Model (k = 6; method: REML)
## 
## Variance Components:
## 
##             estim    sqrt  nlvls  fixed        factor 
## sigma^2.1  0.0000  0.0000      3     no         study 
## sigma^2.2  0.0011  0.0329      6     no  study/result 
## 
## Test for Heterogeneity:
## Q(df = 5) = 4.6596, p-val = 0.4588
## 
## Model Results:
## 
## estimate      se    zval    pval   ci.lb   ci.ub 
##   0.4878  0.1076  4.5335  <.0001  0.2769  0.6987  *** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## $`RVE SEs with Satterthwaite small-sample correction`
## $`RVE SEs with Satterthwaite small-sample correction`$test
##     Coef. Estimate    SE t-stat d.f. p-val (Satt) Sig.
## 1 intrcpt    0.488 0.111   4.41 1.61       0.0701    .
## 
## $`RVE SEs with Satterthwaite small-sample correction`$CIs
##      Coef Estimate    SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt    0.488 0.111 1.61        -0.12          1.1
## 
## 
## $`Prediction interval`
## 95% PI LB 95% PI UB 
##     0.027     0.948 
## 
## $Heterogeneity
##                           Tau                           I^2 
##                    0.03292223                    2.34692161 
##                 Jackson's I^2 Between-cluster heterogeneity 
##                   25.18000000                    0.00000000 
##  Within-cluster heterogeneity                           ICC 
##                    2.35000000                    0.00000000 
## 
## $`Proportion of significant results`
## [1] NA
## 
## $`Publication bias`
## [1] "Publication bias corrections not carried out"
## 
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## [1] "Power for detecting bias-corrected parameter estimates not computed"

Self-regulation

## $`RMA results with model-based SEs`
## 
## Multivariate Meta-Analysis Model (k = 24; method: REML)
## 
## Variance Components:
## 
##             estim    sqrt  nlvls  fixed        factor 
## sigma^2.1  0.0312  0.1765     21     no         study 
## sigma^2.2  0.0436  0.2089     24     no  study/result 
## 
## Test for Heterogeneity:
## Q(df = 23) = 90.2528, p-val < .0001
## 
## Model Results:
## 
## estimate      se    zval    pval   ci.lb   ci.ub 
##   0.3242  0.0716  4.5298  <.0001  0.1839  0.4645  *** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## $`RVE SEs with Satterthwaite small-sample correction`
## $`RVE SEs with Satterthwaite small-sample correction`$test
##     Coef. Estimate     SE t-stat d.f. p-val (Satt) Sig.
## 1 intrcpt    0.324 0.0714   4.54 18.7       <0.001  ***
## 
## $`RVE SEs with Satterthwaite small-sample correction`$CIs
##      Coef Estimate     SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt    0.324 0.0714 18.7        0.175        0.474
## 
## 
## $`Prediction interval`
## 95% PI LB 95% PI UB 
##    -0.265     0.914 
## 
## $Heterogeneity
##                           Tau                           I^2 
##                     0.2734749                    76.4112635 
##                 Jackson's I^2 Between-cluster heterogeneity 
##                    84.2000000                    31.8400000 
##  Within-cluster heterogeneity                           ICC 
##                    44.5800000                     0.4200000 
## 
## $`Proportion of significant results`
## [1] 0.5384615
## 
## $`Publication bias`
## [1] "Publication bias corrections not carried out"
## 
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## [1] "Power for detecting bias-corrected parameter estimates not computed"

Cognitive processes

## $`RMA results with model-based SEs`
## 
## Multivariate Meta-Analysis Model (k = 35; method: REML)
## 
## Variance Components:
## 
##             estim    sqrt  nlvls  fixed        factor 
## sigma^2.1  0.0000  0.0000     29     no         study 
## sigma^2.2  0.0143  0.1194     35     no  study/result 
## 
## Test for Heterogeneity:
## Q(df = 34) = 40.8129, p-val = 0.1959
## 
## Model Results:
## 
## estimate      se     zval    pval   ci.lb   ci.ub 
##   0.5566  0.0485  11.4785  <.0001  0.4616  0.6517  *** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## $`RVE SEs with Satterthwaite small-sample correction`
## $`RVE SEs with Satterthwaite small-sample correction`$test
##     Coef. Estimate     SE t-stat d.f. p-val (Satt) Sig.
## 1 intrcpt    0.557 0.0478   11.6 23.2       <0.001  ***
## 
## $`RVE SEs with Satterthwaite small-sample correction`$CIs
##      Coef Estimate     SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt    0.557 0.0478 23.2        0.458        0.655
## 
## 
## $`Prediction interval`
## 95% PI LB 95% PI UB 
##     0.293     0.820 
## 
## $Heterogeneity
##                           Tau                           I^2 
##                     0.1194099                    19.7930556 
##                 Jackson's I^2 Between-cluster heterogeneity 
##                    26.6100000                     0.0000000 
##  Within-cluster heterogeneity                           ICC 
##                    19.7900000                     0.0000000 
## 
## $`Proportion of significant results`
## [1] 0.75
## 
## $`Publication bias`
## [1] "Publication bias corrections not carried out"
## 
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## [1] "Power for detecting bias-corrected parameter estimates not computed"

Economic decision-making

## $`RMA results with model-based SEs`
## 
## Multivariate Meta-Analysis Model (k = 38; method: REML)
## 
## Variance Components:
## 
##             estim    sqrt  nlvls  fixed        factor 
## sigma^2.1  0.0671  0.2590     25     no         study 
## sigma^2.2  0.0306  0.1750     38     no  study/result 
## 
## Test for Heterogeneity:
## Q(df = 37) = 117.7860, p-val < .0001
## 
## Model Results:
## 
## estimate      se    zval    pval   ci.lb   ci.ub 
##   0.4361  0.0743  5.8697  <.0001  0.2905  0.5818  *** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## $`RVE SEs with Satterthwaite small-sample correction`
## $`RVE SEs with Satterthwaite small-sample correction`$test
##     Coef. Estimate     SE t-stat d.f. p-val (Satt) Sig.
## 1 intrcpt    0.436 0.0743   5.87 22.7       <0.001  ***
## 
## $`RVE SEs with Satterthwaite small-sample correction`$CIs
##      Coef Estimate     SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt    0.436 0.0743 22.7        0.282         0.59
## 
## 
## $`Prediction interval`
## 95% PI LB 95% PI UB 
##    -0.227     1.099 
## 
## $Heterogeneity
##                           Tau                           I^2 
##                      0.312574                     70.182750 
##                 Jackson's I^2 Between-cluster heterogeneity 
##                     81.650000                     48.180000 
##  Within-cluster heterogeneity                           ICC 
##                     22.000000                      0.690000 
## 
## $`Proportion of significant results`
## [1] 0.5581395
## 
## $`Publication bias`
## [1] "Publication bias corrections not carried out"
## 
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## [1] "Power for detecting bias-corrected parameter estimates not computed"

Bias correction results

Emotion

## $`Publication bias`
## $`Publication bias`$`ES-precision correlation`
## [1] 0.2669407
## 
## $`Publication bias`$`4/3PSM`
##    est     se zvalue pvalue   ciLB   ciUB      k  steps 
##  0.260  0.028  9.171  0.000  0.204  0.315 19.000  2.000 
## 
## $`Publication bias`$`PET-PEESE`
## PEESE estimate             se         zvalue         pvalue           ciLB 
##          0.353          0.050          7.110          0.000          0.249 
##           ciUB   PET estimate             se         zvalue         pvalue 
##          0.458          0.301          0.064          4.724          0.000 
##           ciLB           ciUB 
##          0.167          0.436 
## 
## $`Publication bias`$`WAAP-WLS`
##     method term  estimate  std.error  statistic           p.value  conf.low
## 1 WAAP-WLS   b0 0.3105771 0.02925429 0.02925429 0.000000000402088 0.2499074
##   conf.high type kAdequate
## 1 0.3712468    2         1
## 
## $`Publication bias`$`p-uniform*`
##        est       ciLB       ciUB     pvalue 
## 0.30896207 0.19689270 0.41491368 0.01497127 
## 
## $`Publication bias`$`p-curve`
## P-curve analysis 
##  ----------------------- 
## - Total number of provided studies: k = 31 
## - Total number of p<0.05 studies included into the analysis: k = 23 (74.19%) 
## - Total number of studies with p<0.025: k = 14 (45.16%) 
##    
## Results 
##  ----------------------- 
##                     pBinomial  zFull pFull  zHalf pHalf
## Right-skewness test     0.202 -4.994 0.000 -6.551     0
## Flatness test           0.185  1.740 0.959  7.135     1
## Note: p-values of 0 or 1 correspond to p<0.001 and p>0.999, respectively.   
## Power Estimate: 59% (34.7%-78%)
##    
## Evidential value 
##  ----------------------- 
## - Evidential value present: yes 
## - Evidential value absent/inadequate: no 
## 
## 
## $`Power for detecting SESOI and bias-corrected parameter estimates`
##                Median power for detecting a SESOI of d = .20 
##                                                        0.331 
##                Median power for detecting a SESOI of d = .50 
##                                                        0.968 
##                Median power for detecting a SESOI of d = .70 
##                                                        1.000 
## Median power for detecting PET-PEESE estimate.PEESE estimate 
##                                                        0.767 
##               Median power for detecting 4/3PSM estimate.est 
##                                                        0.508

Interpersonal

## $`Publication bias`
## $`Publication bias`$`ES-precision correlation`
## [1] 0.5905207
## 
## $`Publication bias`$`4/3PSM`
##    est     se zvalue pvalue   ciLB   ciUB      k  steps 
##  0.100  0.078  1.295  0.195 -0.052  0.252 56.000  2.000 
## 
## $`Publication bias`$`PET-PEESE`
##   PET estimate             se         zvalue         pvalue           ciLB 
##         -0.215          0.107         -2.000          0.051         -0.430 
##           ciUB PEESE estimate             se         zvalue         pvalue 
##          0.000          0.098          0.077          1.276          0.207 
##           ciLB           ciUB 
##         -0.056          0.252 
## 
## $`Publication bias`$`WAAP-WLS`
##     method term  estimate  std.error  statistic     p.value   conf.low
## 1 WAAP-WLS   b0 0.1388678 0.04068096 0.04068096 0.001043176 0.05780919
##   conf.high type kAdequate
## 1 0.2199264    2         1
## 
## $`Publication bias`$`p-uniform*`
##         est        ciLB        ciUB      pvalue 
##  0.07458773 -0.08100454  0.23262782  0.35226835 
## 
## $`Publication bias`$`p-curve`
## P-curve analysis 
##  ----------------------- 
## - Total number of provided studies: k = 87 
## - Total number of p<0.05 studies included into the analysis: k = 56 (64.37%) 
## - Total number of studies with p<0.025: k = 31 (35.63%) 
##    
## Results 
##  ----------------------- 
##                     pBinomial  zFull pFull  zHalf pHalf
## Right-skewness test     0.252 -4.513 0.000 -6.905     0
## Flatness test           0.008 -0.084 0.466  8.829     1
## Note: p-values of 0 or 1 correspond to p<0.001 and p>0.999, respectively.   
## Power Estimate: 32% (17.9%-49.2%)
##    
## Evidential value 
##  ----------------------- 
## - Evidential value present: yes 
## - Evidential value absent/inadequate: no 
## 
## 
## $`Power for detecting SESOI and bias-corrected parameter estimates`
##              Median power for detecting a SESOI of d = .20 
##                                                    "0.205" 
##              Median power for detecting a SESOI of d = .50 
##                                                    "0.808" 
##              Median power for detecting a SESOI of d = .70 
##                                                    "0.977" 
## Median power for detecting PET-PEESE estimate.PET estimate 
##                    "ES estimate in the opposite direction" 
##             Median power for detecting 4/3PSM estimate.est 
##                                                    "0.087"

Person perception

## $`Publication bias`
## $`Publication bias`$`ES-precision correlation`
## [1] 0.6107588
## 
## $`Publication bias`$`4/3PSM`
##    est     se zvalue pvalue   ciLB   ciUB      k  steps 
##  0.442  0.125  3.528  0.000  0.196  0.687 21.000  2.000 
## 
## $`Publication bias`$`PET-PEESE`
## PEESE estimate             se         zvalue         pvalue           ciLB 
##          0.060          0.050          1.181          0.252         -0.046 
##           ciUB   PET estimate             se         zvalue         pvalue 
##          0.165         -0.236          0.064         -3.710          0.001 
##           ciLB           ciUB 
##         -0.369         -0.103 
## 
## $`Publication bias`$`WAAP-WLS`
##     method term  estimate  std.error  statistic      p.value   conf.low
## 1 WAAP-WLS   b0 0.1963766 0.05084897 0.05084897 0.0004645318 0.09314768
##   conf.high type kAdequate
## 1 0.2996055    2         0
## 
## $`Publication bias`$`p-uniform*`
##       est      ciLB      ciUB    pvalue 
## 0.3497897 0.1311900 0.5620256 0.0055566 
## 
## $`Publication bias`$`p-curve`
## P-curve analysis 
##  ----------------------- 
## - Total number of provided studies: k = 27 
## - Total number of p<0.05 studies included into the analysis: k = 18 (66.67%) 
## - Total number of studies with p<0.025: k = 12 (44.44%) 
##    
## Results 
##  ----------------------- 
##                     pBinomial  zFull pFull  zHalf pHalf
## Right-skewness test     0.119 -5.415 0.000 -6.263     0
## Flatness test           0.413  2.363 0.991  6.659     1
## Note: p-values of 0 or 1 correspond to p<0.001 and p>0.999, respectively.   
## Power Estimate: 69% (44.9%-85.8%)
##    
## Evidential value 
##  ----------------------- 
## - Evidential value present: yes 
## - Evidential value absent/inadequate: no 
## 
## 
## $`Power for detecting SESOI and bias-corrected parameter estimates`
##                Median power for detecting a SESOI of d = .20 
##                                                        0.253 
##                Median power for detecting a SESOI of d = .50 
##                                                        0.899 
##                Median power for detecting a SESOI of d = .70 
##                                                        0.995 
## Median power for detecting PET-PEESE estimate.PEESE estimate 
##                                                        0.067 
##               Median power for detecting 4/3PSM estimate.est 
##                                                        0.816

Self-regulation

## $`Publication bias`
## $`Publication bias`$`ES-precision correlation`
## [1] 0.6350534
## 
## $`Publication bias`$`4/3PSM`
##    est     se zvalue pvalue   ciLB   ciUB      k  steps 
##  0.068  0.114  0.600  0.548 -0.155  0.292 21.000  2.000 
## 
## $`Publication bias`$`PET-PEESE`
##   PET estimate             se         zvalue         pvalue           ciLB 
##         -0.318          0.101         -3.148          0.005         -0.530 
##           ciUB PEESE estimate             se         zvalue         pvalue 
##         -0.107         -0.003          0.056         -0.057          0.955 
##           ciLB           ciUB 
##         -0.120          0.114 
## 
## $`Publication bias`$`WAAP-WLS`
##     method term estimate  std.error  statistic    p.value   conf.low conf.high
## 1 WAAP-WLS   b0   0.1496 0.05771728 0.05771728 0.01630102 0.03020272 0.2689973
##   type kAdequate
## 1    2         1
## 
## $`Publication bias`$`p-uniform*`
##         est        ciLB        ciUB      pvalue 
##  0.12567927 -0.05613884  0.31311091  0.18274946 
## 
## $`Publication bias`$`p-curve`
## P-curve analysis 
##  ----------------------- 
## - Total number of provided studies: k = 35 
## - Total number of p<0.05 studies included into the analysis: k = 24 (68.57%) 
## - Total number of studies with p<0.025: k = 15 (42.86%) 
##    
## Results 
##  ----------------------- 
##                     pBinomial  zFull pFull  zHalf pHalf
## Right-skewness test     0.154 -2.474 0.007 -2.071 0.019
## Flatness test           0.225 -0.642 0.260  4.357 1.000
## Note: p-values of 0 or 1 correspond to p<0.001 and p>0.999, respectively.   
## Power Estimate: 24% (8.1%-48.8%)
##    
## Evidential value 
##  ----------------------- 
## - Evidential value present: yes 
## - Evidential value absent/inadequate: no 
## 
## 
## $`Power for detecting SESOI and bias-corrected parameter estimates`
##              Median power for detecting a SESOI of d = .20 
##                                                    "0.396" 
##              Median power for detecting a SESOI of d = .50 
##                                                    "0.989" 
##              Median power for detecting a SESOI of d = .70 
##                                                        "1" 
## Median power for detecting PET-PEESE estimate.PET estimate 
##                    "ES estimate in the opposite direction" 
##             Median power for detecting 4/3PSM estimate.est 
##                                                    "0.089"

Cognitive processes

## $`Publication bias`
## $`Publication bias`$`ES-precision correlation`
## [1] 0.3800724
## 
## $`Publication bias`$`4/3PSM`
##    est     se zvalue pvalue   ciLB   ciUB      k  steps 
##  0.344  0.129  2.678  0.007  0.092  0.596 28.000  2.000 
## 
## $`Publication bias`$`PET-PEESE`
## PEESE estimate             se         zvalue         pvalue           ciLB 
##          0.405          0.083          4.859          0.000          0.234 
##           ciUB   PET estimate             se         zvalue         pvalue 
##          0.576          0.218          0.146          1.490          0.148 
##           ciLB           ciUB 
##         -0.082          0.519 
## 
## $`Publication bias`$`WAAP-WLS`
##     method term  estimate std.error statistic    p.value  conf.low conf.high
## 1 WAAP-WLS   b0 0.4210579 0.1142019 0.1142019 0.02107484 0.1039825 0.7381332
##   type kAdequate
## 1    1         5
## 
## $`Publication bias`$`p-uniform*`
##        est       ciLB       ciUB     pvalue 
## 0.24115193 0.01326872 0.44907076 0.21851476 
## 
## $`Publication bias`$`p-curve`
## P-curve analysis 
##  ----------------------- 
## - Total number of provided studies: k = 51 
## - Total number of p<0.05 studies included into the analysis: k = 38 (74.51%) 
## - Total number of studies with p<0.025: k = 25 (49.02%) 
##    
## Results 
##  ----------------------- 
##                     pBinomial  zFull pFull  zHalf pHalf
## Right-skewness test     0.036 -5.536  0.00 -6.390     0
## Flatness test           0.273  1.403  0.92  7.798     1
## Note: p-values of 0 or 1 correspond to p<0.001 and p>0.999, respectively.   
## Power Estimate: 50% (30.6%-67.2%)
##    
## Evidential value 
##  ----------------------- 
## - Evidential value present: yes 
## - Evidential value absent/inadequate: no 
## 
## 
## $`Power for detecting SESOI and bias-corrected parameter estimates`
##                Median power for detecting a SESOI of d = .20 
##                                                        0.197 
##                Median power for detecting a SESOI of d = .50 
##                                                        0.789 
##                Median power for detecting a SESOI of d = .70 
##                                                        0.972 
## Median power for detecting PET-PEESE estimate.PEESE estimate 
##                                                        0.609 
##               Median power for detecting 4/3PSM estimate.est 
##                                                        0.476

Economic decision-making

## $`Publication bias`
## $`Publication bias`$`ES-precision correlation`
## [1] 0.6508614
## 
## $`Publication bias`$`4/3PSM`
##    est     se zvalue pvalue   ciLB   ciUB      k  steps 
##  0.234  0.125  1.875  0.061 -0.011  0.478 24.000  2.000 
## 
## $`Publication bias`$`PET-PEESE`
## PEESE estimate             se         zvalue         pvalue           ciLB 
##          0.241          0.134          1.792          0.086         -0.037 
##           ciUB   PET estimate             se         zvalue         pvalue 
##          0.519         -0.135          0.257         -0.526          0.604 
##           ciLB           ciUB 
##         -0.668          0.397 
## 
## $`Publication bias`$`WAAP-WLS`
##     method term  estimate  std.error  statistic       p.value  conf.low
## 1 WAAP-WLS   b0 0.3017689 0.05936583 0.05936583 0.00001090482 0.1814823
##   conf.high type kAdequate
## 1 0.4220555    2         0
## 
## $`Publication bias`$`p-uniform*`
##          est         ciLB         ciUB       pvalue 
##  0.229290001 -0.009305462  0.461452354  0.089691375 
## 
## $`Publication bias`$`p-curve`
## P-curve analysis 
##  ----------------------- 
## - Total number of provided studies: k = 36 
## - Total number of p<0.05 studies included into the analysis: k = 27 (75%) 
## - Total number of studies with p<0.025: k = 14 (38.89%) 
##    
## Results 
##  ----------------------- 
##                     pBinomial  zFull pFull  zHalf pHalf
## Right-skewness test     0.500 -2.332 0.010 -3.352     0
## Flatness test           0.024 -0.937 0.174  4.609     1
## Note: p-values of 0 or 1 correspond to p<0.001 and p>0.999, respectively.   
## Power Estimate: 21% (7.3%-43.7%)
##    
## Evidential value 
##  ----------------------- 
## - Evidential value present: yes 
## - Evidential value absent/inadequate: no 
## 
## 
## $`Power for detecting SESOI and bias-corrected parameter estimates`
##                Median power for detecting a SESOI of d = .20 
##                                                        0.202 
##                Median power for detecting a SESOI of d = .50 
##                                                        0.801 
##                Median power for detecting a SESOI of d = .70 
##                                                        0.976 
## Median power for detecting PET-PEESE estimate.PEESE estimate 
##                                                        0.272 
##               Median power for detecting 4/3PSM estimate.est 
##                                                        0.260

Contour enhanced funnel plots

Forest plots

Moderator/sensitivity analyses

The below reported meta-regressions are all implemented as a multivariate RVE-based models using the CHE working model (Pustejovsky & Tipton, 2020; https://osf.io/preprints/metaarxiv/vyfcj/). Testing of contrasts is carried out using a robust Wald-type test testing the equality of estimates across levels of the moderator.

Moderation by citations and IF

Overall effect moderated by citations and IF

## $test
##                              Coef. Estimate     SE t-stat p-val (z) Sig.
## 1                          intrcpt   0.4209 0.0307  13.70    <0.001  ***
## 2           scale(publicationYear)  -0.0334 0.0293  -1.14    0.2535     
## 3      scale(citationsGSMarch2016)   0.0626 0.0309   2.03    0.0424    *
## 4 scale(h5indexGSJournalMarch2016)  -0.0880 0.0403  -2.18    0.0290    *
## 
## $CIs
##                               Coef Estimate     SE d.f. Lower 95% CI
## 1                          intrcpt   0.4209 0.0307  Inf      0.36071
## 2           scale(publicationYear)  -0.0334 0.0293  Inf     -0.09081
## 3      scale(citationsGSMarch2016)   0.0626 0.0309  Inf      0.00214
## 4 scale(h5indexGSJournalMarch2016)  -0.0880 0.0403  Inf     -0.16688
##   Upper 95% CI
## 1      0.48116
## 2      0.02395
## 3      0.12315
## 4     -0.00903

Moderation by lattitude

Overall effect moderated by lattitude

## $test
##                       Coef. Estimate     SE t-stat p-val (z) Sig.
## 1                   intrcpt   0.4584 0.0367 12.482    <0.001  ***
## 2 scale(latitudeUniversity)   0.0267 0.0362  0.737     0.461     
## 
## $CIs
##                        Coef Estimate     SE d.f. Lower 95% CI Upper 95% CI
## 1                   intrcpt   0.4584 0.0367  Inf       0.3864       0.5304
## 2 scale(latitudeUniversity)   0.0267 0.0362  Inf      -0.0442       0.0975

Compensatory effects moderated by lattitude

## $test
##                       Coef. Estimate     SE t-stat p-val (z) Sig.
## 1                   intrcpt   0.3051 0.0516  5.913    <0.001  ***
## 2 scale(latitudeUniversity)  -0.0323 0.0528 -0.612     0.541     
## 
## $CIs
##                        Coef Estimate     SE d.f. Lower 95% CI Upper 95% CI
## 1                   intrcpt   0.3051 0.0516  Inf        0.204       0.4063
## 2 scale(latitudeUniversity)  -0.0323 0.0528  Inf       -0.136       0.0712

Priming effects moderated by lattitude

## $test
##                       Coef. Estimate    SE t-stat p-val (z) Sig.
## 1                   intrcpt   0.5120 0.037  13.86    <0.001  ***
## 2 scale(latitudeUniversity)   0.0451 0.037   1.22     0.223     
## 
## $CIs
##                        Coef Estimate    SE d.f. Lower 95% CI Upper 95% CI
## 1                   intrcpt   0.5120 0.037  Inf       0.4396        0.584
## 2 scale(latitudeUniversity)   0.0451 0.037  Inf      -0.0274        0.118

Mood effects moderated by lattitude

## $test
##                       Coef. Estimate     SE t-stat p-val (z) Sig.
## 1                   intrcpt    0.232 0.0463  5.017    <0.001  ***
## 2 scale(latitudeUniversity)   -0.016 0.0353 -0.451     0.652     
## 
## $CIs
##                        Coef Estimate     SE d.f. Lower 95% CI Upper 95% CI
## 1                   intrcpt    0.232 0.0463  Inf       0.1417       0.3233
## 2 scale(latitudeUniversity)   -0.016 0.0353  Inf      -0.0852       0.0533

Moderation by gender

Overall effect moderated by gender ratio

## $test
##               Coef. Estimate     SE t-stat p-val (z) Sig.
## 1           intrcpt    0.427 0.0338  12.63   < 0.001  ***
## 2 scale(percFemale)    0.103 0.0388   2.64   0.00821   **
## 
## $CIs
##                Coef Estimate     SE d.f. Lower 95% CI Upper 95% CI
## 1           intrcpt    0.427 0.0338  Inf       0.3609        0.494
## 2 scale(percFemale)    0.103 0.0388  Inf       0.0266        0.179

Compensatory effects moderated by gender ratio

## $test
##               Coef. Estimate     SE t-stat p-val (z) Sig.
## 1           intrcpt   0.3062 0.0494  6.204    <0.001  ***
## 2 scale(percFemale)  -0.0249 0.0568 -0.438     0.661     
## 
## $CIs
##                Coef Estimate     SE d.f. Lower 95% CI Upper 95% CI
## 1           intrcpt   0.3062 0.0494  Inf        0.209       0.4030
## 2 scale(percFemale)  -0.0249 0.0568  Inf       -0.136       0.0864

Priming effects moderated by gender ratio

## $test
##               Coef. Estimate     SE t-stat p-val (z) Sig.
## 1           intrcpt    0.467 0.0352   13.3    <0.001  ***
## 2 scale(percFemale)    0.148 0.0422    3.5    <0.001  ***
## 
## $CIs
##                Coef Estimate     SE d.f. Lower 95% CI Upper 95% CI
## 1           intrcpt    0.467 0.0352  Inf        0.398        0.536
## 2 scale(percFemale)    0.148 0.0422  Inf        0.065        0.231

Published status

## $`Model results`
## $`Model results`$test
##                Coef. Estimate     SE t-stat p-val (z) Sig.
## 1 factor(published)0    0.312 0.0773   4.03    <0.001  ***
## 2 factor(published)1    0.432 0.0347  12.47    <0.001  ***
## 
## $`Model results`$CIs
##                 Coef Estimate     SE d.f. Lower 95% CI Upper 95% CI
## 1 factor(published)0    0.312 0.0773  Inf        0.160        0.463
## 2 factor(published)1    0.432 0.0347  Inf        0.364        0.500
## 
## 
## $`RVE Wald test`
##  test Fstat df_num df_denom p_val sig
##   HTZ  2.05      1     24.3 0.164

Comparing randomized and non-randomized designs

## $`Model results`
## $`Model results`$test
##          Coef. Estimate     SE t-stat p-val (z) Sig.
## 1 factor(rct)0    0.260 0.0543   4.79    <0.001  ***
## 2 factor(rct)1    0.447 0.0360  12.40    <0.001  ***
## 
## $`Model results`$CIs
##           Coef Estimate     SE d.f. Lower 95% CI Upper 95% CI
## 1 factor(rct)0    0.260 0.0543  Inf        0.154        0.367
## 2 factor(rct)1    0.447 0.0360  Inf        0.376        0.517
## 
## 
## $`RVE Wald test`
##  test Fstat df_num df_denom   p_val sig
##   HTZ  8.72      1     34.7 0.00562  **

Subgroups of observational and randomized effects

##                k   g [95% CI]        SE   Tau  I^2 3PSM est [95% CI] 
## Non.randomized 34  0.26 [0.14, 0.37] 0.06 0.22 80% 0.09 [-0.02, 0.21]
## Randomized     148 0.45 [0.38, 0.52] 0.04 0.29 70% 0.22 [0.1, 0.35]  
##                3PSM.pvalue PET-PEESE est [95% CI] PET-PEESE.pvalue
## Non.randomized 0.117       -0.1 [-0.31, 0.12]     0.358           
## Randomized     0.001       0.19 [0.04, 0.33]      0.011

Non-randomized

## $`RMA results with model-based SEs`
## 
## Multivariate Meta-Analysis Model (k = 29; method: REML)
## 
## Variance Components:
## 
##             estim    sqrt  nlvls  fixed        factor 
## sigma^2.1  0.0387  0.1968     24     no         study 
## sigma^2.2  0.0116  0.1078     29     no  study/result 
## 
## Test for Heterogeneity:
## Q(df = 28) = 102.6511, p-val < .0001
## 
## Model Results:
## 
## estimate      se    zval    pval   ci.lb   ci.ub 
##   0.2554  0.0567  4.5059  <.0001  0.1443  0.3665  *** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## $`RVE SEs with Satterthwaite small-sample correction`
## $`RVE SEs with Satterthwaite small-sample correction`$test
##     Coef. Estimate     SE t-stat d.f. p-val (Satt) Sig.
## 1 intrcpt    0.255 0.0564   4.52 21.1       <0.001  ***
## 
## $`RVE SEs with Satterthwaite small-sample correction`$CIs
##      Coef Estimate     SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt    0.255 0.0564 21.1        0.138        0.373
## 
## 
## $`Prediction interval`
## 95% PI LB 95% PI UB 
##    -0.223     0.734 
## 
## $Heterogeneity
##                           Tau                           I^2 
##                     0.2243704                    79.6290061 
##                 Jackson's I^2 Between-cluster heterogeneity 
##                    89.8100000                    61.2600000 
##  Within-cluster heterogeneity                           ICC 
##                    18.3700000                     0.7700000 
## 
## $`Proportion of significant results`
## [1] 0.3823529
## 
## $`Publication bias`
## $`Publication bias`$`ES-precision correlation`
## [1] 0.4714771
## 
## $`Publication bias`$`4/3PSM`
##    est     se zvalue pvalue   ciLB   ciUB      k  steps 
##  0.092  0.059  1.569  0.117 -0.023  0.208 24.000  2.000 
## 
## $`Publication bias`$`PET-PEESE`
##   PET estimate             se         zvalue         pvalue           ciLB 
##         -0.097          0.104         -0.938          0.358         -0.312 
##           ciUB PEESE estimate             se         zvalue         pvalue 
##          0.118          0.094          0.061          1.536          0.139 
##           ciLB           ciUB 
##         -0.033          0.222 
## 
## $`Publication bias`$`WAAP-WLS`
##     method term  estimate  std.error  statistic   p.value   conf.low conf.high
## 1 WAAP-WLS   b0 0.1493071 0.09129957 0.09129957 0.2436033 -0.2435232 0.5421375
##   type kAdequate
## 1    1         3
## 
## $`Publication bias`$`p-uniform*`
##        est       ciLB       ciUB     pvalue 
## 0.14359084 0.01474893 0.27467682 0.03523204 
## 
## $`Publication bias`$`p-curve`
## P-curve analysis 
##  ----------------------- 
## - Total number of provided studies: k = 23 
## - Total number of p<0.05 studies included into the analysis: k = 11 (47.83%) 
## - Total number of studies with p<0.025: k = 7 (30.43%) 
##    
## Results 
##  ----------------------- 
##                     pBinomial  zFull pFull  zHalf pHalf
## Right-skewness test     0.274 -3.271 0.001 -3.501     0
## Flatness test           0.389  1.111 0.867  4.880     1
## Note: p-values of 0 or 1 correspond to p<0.001 and p>0.999, respectively.   
## Power Estimate: 58% (22.5%-83.7%)
##    
## Evidential value 
##  ----------------------- 
## - Evidential value present: yes 
## - Evidential value absent/inadequate: no 
## 
## 
## $`Power for detecting SESOI and bias-corrected parameter estimates`
##              Median power for detecting a SESOI of d = .20 
##                                                    "0.367" 
##              Median power for detecting a SESOI of d = .50 
##                                                    "0.982" 
##              Median power for detecting a SESOI of d = .70 
##                                                        "1" 
## Median power for detecting PET-PEESE estimate.PET estimate 
##                    "ES estimate in the opposite direction" 
##             Median power for detecting 4/3PSM estimate.est 
##                                                    "0.116"

Randomized

## $`RMA results with model-based SEs`
## 
## Multivariate Meta-Analysis Model (k = 144; method: REML)
## 
## Variance Components:
## 
##             estim    sqrt  nlvls  fixed        factor 
## sigma^2.1  0.0464  0.2155     92     no         study 
## sigma^2.2  0.0364  0.1907    144     no  study/result 
## 
## Test for Heterogeneity:
## Q(df = 143) = 559.4052, p-val < .0001
## 
## Model Results:
## 
## estimate      se     zval    pval   ci.lb   ci.ub 
##   0.4521  0.0364  12.4333  <.0001  0.3809  0.5234  *** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## $`RVE SEs with Satterthwaite small-sample correction`
## $`RVE SEs with Satterthwaite small-sample correction`$test
##     Coef. Estimate     SE t-stat d.f. p-val (Satt) Sig.
## 1 intrcpt    0.452 0.0364   12.4 82.9       <0.001  ***
## 
## $`RVE SEs with Satterthwaite small-sample correction`$CIs
##      Coef Estimate     SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt    0.452 0.0364 82.9         0.38        0.525
## 
## 
## $`Prediction interval`
## 95% PI LB 95% PI UB 
##    -0.124     1.028 
## 
## $Heterogeneity
##                           Tau                           I^2 
##                     0.2877887                    69.8305082 
##                 Jackson's I^2 Between-cluster heterogeneity 
##                    80.4600000                    39.1500000 
##  Within-cluster heterogeneity                           ICC 
##                    30.6800000                     0.5600000 
## 
## $`Proportion of significant results`
## [1] 0.6351351
## 
## $`Publication bias`
## $`Publication bias`$`ES-precision correlation`
## [1] 0.5386684
## 
## $`Publication bias`$`4/3PSM`
##    est     se zvalue pvalue   ciLB   ciUB      k  steps 
##  0.220  0.064  3.443  0.001  0.095  0.346 92.000  2.000 
## 
## $`Publication bias`$`PET-PEESE`
## PEESE estimate             se         zvalue         pvalue           ciLB 
##          0.186          0.072          2.586          0.011          0.043 
##           ciUB   PET estimate             se         zvalue         pvalue 
##          0.329         -0.145          0.096         -1.511          0.134 
##           ciLB           ciUB 
##         -0.336          0.046 
## 
## $`Publication bias`$`WAAP-WLS`
##     method term    estimate std.error statistic   p.value   conf.low conf.high
## 1 WAAP-WLS   b0 -0.02180643 0.1306439 0.1306439 0.8729201 -0.3414805 0.2978676
##   type kAdequate
## 1    1         7
## 
## $`Publication bias`$`p-uniform*`
##         est        ciLB        ciUB      pvalue 
## 0.205756568 0.082334253 0.325994919 0.006398717 
## 
## $`Publication bias`$`p-curve`
## P-curve analysis 
##  ----------------------- 
## - Total number of provided studies: k = 140 
## - Total number of p<0.05 studies included into the analysis: k = 98 (70%) 
## - Total number of studies with p<0.025: k = 63 (45%) 
##    
## Results 
##  ----------------------- 
##                     pBinomial  zFull pFull  zHalf pHalf
## Right-skewness test     0.003 -8.542  0.00 -9.642     0
## Flatness test           0.076  1.887  0.97 12.097     1
## Note: p-values of 0 or 1 correspond to p<0.001 and p>0.999, respectively.   
## Power Estimate: 47% (35.1%-58.7%)
##    
## Evidential value 
##  ----------------------- 
## - Evidential value present: yes 
## - Evidential value absent/inadequate: no 
## 
## 
## $`Power for detecting SESOI and bias-corrected parameter estimates`
##                Median power for detecting a SESOI of d = .20 
##                                                        0.204 
##                Median power for detecting a SESOI of d = .50 
##                                                        0.806 
##                Median power for detecting a SESOI of d = .70 
##                                                        0.977 
## Median power for detecting PET-PEESE estimate.PEESE estimate 
##                                                        0.183 
##               Median power for detecting 4/3PSM estimate.est 
##                                                        0.237

F-test of equality of variances

Mean vi for non-randomized designs

## [1] 0.04069351

Mean vi for randomized designs

## [1] 0.0660096

F-statistics

## [1] 0.6164787

F-test p-value

## [1] 0.1272321

Comparing effects based on student and non-student samples

## $`Model results`
## $`Model results`$test
##                    Coef. Estimate     SE t-stat p-val (z) Sig.
## 1 factor(studentSample)0    0.327 0.0456   7.16    <0.001  ***
## 2 factor(studentSample)1    0.453 0.0395  11.46    <0.001  ***
## 
## $`Model results`$CIs
##                     Coef Estimate     SE d.f. Lower 95% CI Upper 95% CI
## 1 factor(studentSample)0    0.327 0.0456  Inf        0.237        0.416
## 2 factor(studentSample)1    0.453 0.0395  Inf        0.375        0.530
## 
## 
## $`RVE Wald test`
##  test Fstat df_num df_denom  p_val sig
##   HTZ  4.36      1       81 0.0398   *

Year of Publication

Linear mixed-effects model. Taking into effect clustering of ESs due to originating from the same study. Using square root of variance to make the distribution normal.

##                                    Estimate Std. Error       df   t value
## (Intercept)                      -0.1407496 0.09437155 85.97906 -1.491442
## scale(h5indexGSJournalMarch2016) -0.1697075 0.10151050 85.90369 -1.671823
## scale(publicationYear)           -0.2307599 0.08925779 85.99053 -2.585320
##                                    Pr(>|t|)
## (Intercept)                      0.13950560
## scale(h5indexGSJournalMarch2016) 0.09819827
## scale(publicationYear)           0.01141363

Comment: all the variables were centered for easier interpretation of model coefficients. See the negative beta for Publication Year. The more recent a publication, the lower the variance (better precision), controlling for H5.

Scatterplot year <-> precision

Size of the points indicate the H5 index (the bigger the higher) of the journal that the ES is published in.

## `geom_smooth()` using formula 'y ~ x'

Citations

Linear mixed-effects model. Taking into effect clustering of ESs due to originating from the same study. Using square root of variance to make the distribution normal.

##                                      Estimate Std. Error       df     t value
## (Intercept)                      -0.201059160 0.09182019 84.97688 -2.18970532
## scale(publicationYear)           -0.003047723 0.11144001 85.08921 -0.02734855
## scale(h5indexGSJournalMarch2016) -0.365198106 0.11472691 84.73581 -3.18319482
## scale(citationsGSMarch2016)       0.332488656 0.10530882 85.10522  3.15727251
##                                     Pr(>|t|)
## (Intercept)                      0.031286371
## scale(publicationYear)           0.978245772
## scale(h5indexGSJournalMarch2016) 0.002037484
## scale(citationsGSMarch2016)      0.002203503

Scatterplot precision <-> citations

The relationship between precision (sqrt of variance) and number of citations.

## `geom_smooth()` using formula 'y ~ x'

H5 index

Linear mixed-effects model. Taking into effect clustering of ESs due to originating from the same study. Using square root of variance to make the distribution normal.

##                                    Estimate Std. Error       df   t value
## (Intercept)                      -0.1141591  0.0968350 87.04277 -1.178903
## scale(h5indexGSJournalMarch2016) -0.1178194  0.1027219 86.98377 -1.146974
##                                   Pr(>|t|)
## (Intercept)                      0.2416493
## scale(h5indexGSJournalMarch2016) 0.2545376

Scatterplot precision <-> journal H5

The relationship between precision (sqrt of variance) and H5 index of the journal.

## `geom_smooth()` using formula 'y ~ x'

Decline effect

Linear mixed-effects model. Taking into effect clustering of ESs due to originating from the same study.

##                           Estimate Std. Error       df  t value      Pr(>|t|)
## (Intercept)            0.001256607 0.08446073 86.44516 0.014878 0.98816379458
## scale(sqrt(vi))        0.361986905 0.08744592 79.13890 4.139552 0.00008628732
## scale(publicationYear) 0.114376928 0.08487961 97.53930 1.347519 0.18093479180

Citation bias

Do more highly-cited studies report larger effect sizes?

##                                     Estimate Std. Error       df    t value
## (Intercept)                       0.22906790 0.01409808 76.12334 16.2481604
## scale(publicationYear)           -0.01667034 0.01740063 82.82924 -0.9580309
## scale(h5indexGSJournalMarch2016) -0.04319160 0.01701388 64.19127 -2.5386092
## scale(citationsGSMarch2016)       0.02599613 0.01651275 85.24009  1.5743067
##                                                            Pr(>|t|)
## (Intercept)                      0.00000000000000000000000001855457
## scale(publicationYear)           0.34083527493986021106309181050165
## scale(h5indexGSJournalMarch2016) 0.01356484913606161198107447063421
## scale(citationsGSMarch2016)      0.11912106790969252678724643601527

P-curve for interaction effects

## P-curve analysis 
##  ----------------------- 
## - Total number of provided studies: k = 51 
## - Total number of p<0.05 studies included into the analysis: k = 37 (72.55%) 
## - Total number of studies with p<0.025: k = 22 (43.14%) 
##    
## Results 
##  ----------------------- 
##                     pBinomial  zFull pFull  zHalf pHalf
## Right-skewness test     0.162 -4.955 0.000 -6.205     0
## Flatness test           0.080  0.988 0.838  7.763     1
## Note: p-values of 0 or 1 correspond to p<0.001 and p>0.999, respectively.   
## Power Estimate: 45% (25.8%-64.1%)
##    
## Evidential value 
##  ----------------------- 
## - Evidential value present: yes 
## - Evidential value absent/inadequate: no

P-curve and contour-enhanced funnel plots

Counts

Simple counts: 1. How often did authors test for moderation by attachment?

## 
##   0   1 
## 303  19
  1. Via validated tests: How often was tested for:
  1. Independence of awareness
## 
##   0 
## 323
  1. Lack of intention
## 
##   0 
## 320
  1. Efficiency of behavior
## 
##   0 
## 317
  1. Lack of control of behavior
## 
##   0 
## 321
  1. Via non-validated tests: How often was tested for:
  1. Independence of awareness
## 
##         0         1 
## 0.7080925 0.2167630
## [1] 51
  1. Lack of intention
## 
##   0 
## 323
  1. Efficiency of behavior
## 
##   0 
## 320
  1. Lack of control of behavior
## 
##   0 
## 322
  1. Whether researchers tested for innateness
## 
##   0 
## 320
  1. Skin temperature location
## 
## 1 2 
## 6 5
  1. Population type
## 
##         general special student 
##      28      84       8     226
  1. Control group type (1 = active controls)
## 
##  0  1 
##  6 37
  1. In how many countries the studies were conducted and what their mean distance was from the equator
##  [1] "USA"         ""            "China"       "Portugal"    "Singapore"  
##  [6] "Israel"      "Germany"     "South Korea" "Netherlands" "Japan"      
## [11] "Scotland"    "England"     "India"       "Canada"      "Switzerland"
## [16] "Poland"      "Italy"
  1. Number of independent studies
## [1] 84
  1. Number of papers
## [1] 33
  1. Lattitude
## Lattitude mean   Lattitude SD            Min            Max 
##       39.74729       10.84424        1.29686       57.16498

Further sensitivity analyses

Different step functions for the 3PSM

##    steps deltaModerate deltaSevere deltaExtreme
## 1 0.0025          1.00        1.00         1.00
## 2 0.0050          0.99        0.99         0.98
## 3 0.0125          0.97        0.97         0.95
## 4 0.0250          0.95        0.95         0.90
## 5 0.0500          0.80        0.65         0.50
## 6 0.1000          0.60        0.40         0.20
## 7 0.2500          0.50        0.25         0.10
## 8 0.5000          0.50        0.25         0.10
## 9 1.0000          0.50        0.25         0.10
## $deltaModerate
##     est      se  zvalue  pvalue    ciLB    ciUB       k   steps 
##   0.320   0.032  10.115   0.000   0.258   0.383 114.000   9.000 
## 
## $deltaSevere
##     est      se  zvalue  pvalue    ciLB    ciUB       k   steps 
##   0.236   0.036   6.613   0.000   0.166   0.306 116.000   9.000 
## 
## $deltaExtreme
##     est      se  zvalue  pvalue    ciLB    ciUB       k   steps 
##   0.134   0.031   4.295   0.000   0.073   0.195 116.000   9.000
## R version 4.0.2 (2020-06-22)
## Platform: x86_64-apple-darwin17.0 (64-bit)
## Running under: macOS  10.16
## 
## Matrix products: default
## BLAS:   /Library/Frameworks/R.framework/Versions/4.0/Resources/lib/libRblas.dylib
## LAPACK: /Library/Frameworks/R.framework/Versions/4.0/Resources/lib/libRlapack.dylib
## 
## locale:
## [1] sk_SK.UTF-8/sk_SK.UTF-8/sk_SK.UTF-8/C/sk_SK.UTF-8/sk_SK.UTF-8
## 
## attached base packages:
## [1] stats     graphics  grDevices utils     datasets  methods   base     
## 
## other attached packages:
##  [1] poibin_1.5         ddpcr_1.15         clubSandwich_0.5.2 weightr_2.0.2     
##  [5] scales_1.1.1       magrittr_2.0.1     multcomp_1.4-13    TH.data_1.0-10    
##  [9] MASS_7.3-51.6      survival_3.1-12    mvtnorm_1.1-1      Amelia_1.7.6      
## [13] Rcpp_1.0.6         pwr_1.3-0          lmerTest_3.1-2     kableExtra_1.3.1  
## [17] puniform_0.2.2     knitr_1.30         lme4_1.1-26        esc_0.5.1         
## [21] dmetar_0.0.9000    meta_4.16-2        metafor_2.5-75     Matrix_1.2-18     
## [25] psych_2.0.7        forcats_0.5.0      stringr_1.4.0      dplyr_1.0.1       
## [29] purrr_0.3.4        readr_1.3.1        tidyr_1.1.1        tibble_3.0.6      
## [33] ggplot2_3.3.3      tidyverse_1.3.0    reshape_0.8.8      car_3.0-9         
## [37] carData_3.0-4     
## 
## loaded via a namespace (and not attached):
##  [1] minqa_1.2.4         colorspace_2.0-0    ellipsis_0.3.1     
##  [4] class_7.3-17        modeltools_0.2-23   rio_0.5.16         
##  [7] mclust_5.4.7        fs_1.5.0            rstudioapi_0.13    
## [10] farver_2.0.3        ggrepel_0.9.1       flexmix_2.3-17     
## [13] lubridate_1.7.9     mathjaxr_1.2-0      xml2_1.3.2         
## [16] codetools_0.2-16    splines_4.0.2       mnormt_2.0.1       
## [19] robustbase_0.93-7   jsonlite_1.7.2      nloptr_1.2.2.2     
## [22] broom_0.7.0         cluster_2.1.0       kernlab_0.9-29     
## [25] dbplyr_1.4.4        compiler_4.0.2      httr_1.4.2         
## [28] backports_1.1.8     assertthat_0.2.1    cli_2.3.0          
## [31] htmltools_0.5.0     tools_4.0.2         gtable_0.3.0       
## [34] glue_1.4.2          cellranger_1.1.0    vctrs_0.3.6        
## [37] nlme_3.1-148        fpc_2.2-9           xfun_0.19          
## [40] openxlsx_4.1.5      rvest_0.3.6         CompQuadForm_1.4.3 
## [43] lifecycle_0.2.0     statmod_1.4.35      DEoptimR_1.0-8     
## [46] zoo_1.8-8           hms_0.5.3           sandwich_2.5-1     
## [49] parallel_4.0.2      yaml_2.2.1          curl_4.3           
## [52] gridExtra_2.3       MuMIn_1.43.17       stringi_1.5.3      
## [55] highr_0.8           boot_1.3-25         zip_2.1.0          
## [58] rlang_0.4.10        pkgconfig_2.0.3     prabclus_2.3-2     
## [61] evaluate_0.14       lattice_0.20-41     labeling_0.4.2     
## [64] tidyselect_1.1.0    plyr_1.8.6          R6_2.5.0           
## [67] generics_0.0.2      DBI_1.1.0           mgcv_1.8-31        
## [70] pillar_1.4.7        haven_2.3.1         foreign_0.8-80     
## [73] withr_2.4.1         abind_1.4-5         nnet_7.3-14        
## [76] modelr_0.1.8        crayon_1.4.0        tmvnsim_1.0-2      
## [79] rmarkdown_2.5.3     grid_4.0.2          readxl_1.3.1       
## [82] data.table_1.13.0   netmeta_1.3-0       blob_1.2.1         
## [85] webshot_0.5.2       reprex_0.3.0        digest_0.6.27      
## [88] diptest_0.75-7      numDeriv_2016.8-1.1 stats4_4.0.2       
## [91] munsell_0.5.0       viridisLite_0.3.0   magic_1.5-9